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Background / Context: 

Over the last decade, New York City (NYC) has been the site of a systemwide high school 
reform effort that is unprecedented in its scope and pace. Since 2002, the school district has 
closed more than 20 failing high schools, opened more than 200 new secondary schools, and 
implemented a centralized high school admission process in which approximately 80,000 
students a year indicate their school preferences from a wide-ranging choice of programs. 

At the heart of these reforms lie the new schools that are often referred to as “small 
schools of choice” (SSCs) — small, academically nonselective, public high schools that were 
opened between 2002 and 2008. Serving approximately 100 students per grade in grades 9 
through 12 and open to students at all levels of academic achievement, the SSCs were created to 
serve the district’s most disadvantaged and historically underserved students. 

By taking advantage of a naturally-occurring lotteries in the NYC Department of 
Education’s high school application process, MDRC researchers are able to estimate the effects 
of enrolling in SSCs on students’ future academic outcomes using a sample of over 20,000 
students. The MDRC project team published its first report on these findings in June 2010 and 
has been funded by the Bill and Melinda Gates Foundation to publish annual reports through 
2014. In these annual reports MDRC will follow students in the sample into college and 
community college; collect data from those in the field on what they perceive to be key features 
of the SSC theory of action and sources of the SSC effects; explore variation in the lottery 
impacts by school characteristics related to the SSC theory of action; and describe SSC teachers 
in greater detail. In this proposed panel, Rebecca Unterman, an author on MDRC’s small schools 
of choice studies, will share the project’s most recent findings and discuss the team’s experiences 
working to bridge the gap between policy and practice in New York. The other panelists, 
experienced policymakers in NYC, will provide their perspective on the effects of the study (and 
other research) on their practice. 

Purpose / Objective / Research Question / Focus of Study: 

MDRC’s NYC Small Schools of Choice project has provided MDRC researchers and 
NYC policymakers with a unique opportunity to integrate rigorous research and practice over an 
extended period of time. The project’s rigorous methodology has given MDRC researchers the 
chance to answer pressing questions for education reformers about the effectiveness of a large- 
scale NYC DOE intervention, while at the same time, it’s given policymakers like Saskia 
Thompson at the NYC DOE and Robert Hughes at New Visions for Public Schools, the 
opportunity to inform the research design and ensure that the findings are meaningful to 
policymakers and practitioners. The bullets below list the proposed panel participants and 
describe their professional backgrounds as they relate to the NYC SSC reforms and the topics 
that they will discuss on the proposed panel. 

• Robert Hughes is the president of New Visions for Public Schools. Through the New 
Century High School Initiative New Visions was responsible for the creation of the first 
SSCs in New York and has since continued to create and support NYC’s small high schools 
of choice. 

• Saskia Thompson is the Deputy Chief Schools Officer at the New York City Department of 
Education (NYC DOE). In her work she is responsible for implementing school policies 
across the district. 
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• Rebecca Unterman is a Research Associate in the K-12 group at MDRC and is a lead 
researcher on the NYC Small Schools of Choice study. 

• David Silver ( moderator j is a Senior Program Officer at the Bill and Melinda Gates 
Foundation. In this role he oversees the MDRC NYC Small Schools of Choice project, as 
well as other large-scale evaluations funded by the BMGF. 

Together this panel is uniquely positioned to answer key questions about the design of the 
MDRC study, the development of the intervention at the district level, and their perspective on 
how to best integrate rigorous research and practice. During the session, the moderator, David 
Silver, will loosely follow the following format: 

• 10 minutes: Introductions and framing by the panel moderator. 

• 20 minutes: Robert Hughes and Saskia Thompson will share their perspective on the creation 
of NYC’s small schools of choice and the role of research in their initial NYC work 

• 20 minutes: Rebecca Untennan will share the most recent findings from the MDRC small 
schools of choice project and field questions about the study 

• 40 minutes: The moderator will invite questions and lead a discussion on the panelists’ 
experience bridging research and practice in New York and discuss the implications of this 
work in other settings. 

Setting: 

The New York City public school system is the largest in the United States, with over 1 . 1 
million students enrolled in more than 1,600 schools. Over the past decade, it has been the site of 
an ambitious effort to reform the high school system, of which the creation of SSCs was a central 
part. Beginning in 2002, the New York City Department of Education (DOE) accelerated and 
expanded efforts that had been under way since the mid-1990s to close large, low-perfonning 
schools and open new small schools in their stead. These reform efforts were supported by a 
consortium of funders led by the Bill & Melinda Gates Foundation — which ultimately invested 
over $150 million in New York City 1 — and were implemented in partnership with the teachers 
and principals unions. 2 

Population / Participants / Subjects: 

SSCs are primarily located in some of the most economically disadvantaged areas of 
New York City. The majority of students in the study’s analysis sample enter high school 
performing below grade-level in either reading or math and qualify for free or reduced-price 
lunch. Specifically, 70.5 percent of the sample performed below grade-level in reading in 8 th 
grade; 63.9 percent of the sample performed below grade-level in math; and 83.9 percent of the 
sample qualified for free or reduced-price lunch. Forty-seven percent of the student sample is 
Black and 44 percent is Hispanic. 


1 The Gates Foundation supported the DOE’s new school creation efforts in partnership with the Carnegie 
Corporation of New York and the Open Society Institute, and other systemwide initiatives benefited from at least 
$230 million worth of funding from philanthropies including the Wallace Foundation, the Michael & Susan Dell 
Foundation, and the Eli and Edythe Broad Foundation. Quint, Smith, Unterman, and Moedano (2010) provides a 
history of small schools in New York City, including the efforts undertaken by New Visions for Public Schools — 
which launched the New Century High Schools Initiative — that immediately preceded and served as the model for 
the school creation efforts under the Bloomberg/Klein administration. 

2 New Visions for Public Schools (2005). 
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Intervention / Program / Practice: 

SSCs were developed through a competitive proposal process that was designed to both 
stimulate innovative ideas from a range of stakeholders and ensure that school founders met pre- 
specified conditions regarding academic rigor and sustained personal relationships among 
students and faculty. In addition, most SSCs were founded with community partners who offer 
students relevant learning opportunities inside and outside the classroom and provide school 
faculty with additional staffing support and resources during start-up. 3 By integrating a 
demanding and comprehensive academic curriculum, personal attention to student academic 
progress, and real-world experiences with community partners, the new small schools seek to 
prepare students for college and career. 

Unlike in some cities, where small high schools are created by reconfiguring large 
existing schools into smaller units in the same buildings with the same teachers and students; the 
typical SSC was created “from scratch” with a principal, teachers, and students who were new to 
the school. It is important to note however, that SSCs are not just small in size. They are 
purposefully organized around smaller educational units that are designed to give students a 
better chance of being known by their teachers and other adults in the building. For example, 
many SSCs have student advisory structures that carve out time at least once a week for teachers 
to meet with students and discuss youth development issues such as maintaining self-esteem, 
building supportive peer relationships and facing adversity. In addition, many SSCs organize 
teachers into grade-level teams and provide common planning time for teachers to discuss 
students’ progress and solve problems for students who are struggling. 

Research Design: 

In the spring of 2004, the city introduced the High School Application Processing System 
(HSAPS), a centralized choice process that was to govern the placement of all entering ninth- 
grade students. HSAPS uses an objective, computer-based process to assign about 72,500 
entering ninth-graders annually to about 400 public high schools. When they are in the eighth 
grade, students who participate in HSAPS indicate, in order of preference, up to 12 high schools 
they would like to attend. Each year, some schools have more applicants than seats available. 
When this occurs at an SSC, a lottery is created within HSAPS that randomly determines which 
students are assigned to that school. 

The existence of these lotteries provides an unprecedented opportunity to launch a 
rigorous study of the effects of this group of schools on student academic achievement, because 
the lotteries create two randomized groups among students who chose a given SSC — those who 
won its lottery and were assigned to the SSC and those who lost its lottery and were assigned 
elsewhere. Future outcomes for these two groups can be compared to obtain valid estimates of 
the effects of SSCs on student achievement. The lotteries created by HSAPS, together with the 
unusually large size of the randomized sample they produced, allow for a high degree of validity 
and precision in the present analyses. Thus, one can have considerable confidence in them. Using 
these lotteries as the basis for its analysis, this report presents the estimated effects of enrolling 
in a small school of choice versus enrolling in one of the other high schools that are available to 
the average incoming ninth-grader. 4 


3 In their first two years of operation SSCs created in the early years of this district-wide reform effort were permitted by the New York City 
Department of Education to exclude English Language Learners and special education students requiring resource room accommodations. As a 
result of a court decision, the New York City Department of Education ended this practice. 

4 As explained in Appendix A of MDRC’s June 2010 report by Bloom, Thompson and Unterman, to estimate the 
effects of enrolling in an SSC, the estimated effects of winning an SSC lottery are adjusted to account for the 
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Data Collection and Analysis: 

The analysis presented at this conference will draw on data from NYC school district’s 
high school admissions process to identify a sample of students who chose SSCs, but who — 
because their chosen SSC had more applicants than seats available — were assigned via lottery 
either to that school or to a subsequent choice on their list. The analysis includes four annual 
cohorts of students who entered high school in the fall of 2005, 2006, 2007, and 2008, 
respectively — a total of 21,085 students who applied to the 105 SSCs that were oversubscribed, 
and for which lotteries were held, during the study period. MDRC researchers have collected 
student enrollment and high school graduation data for these students through the 2009-2010 
school year, thus they are able to report four-year high school graduation rates for the first two 
cohorts. 

In an effort to leam more about the key features of SSCs, in 201 1 MDRC partnered with 
the Research Alliance for New York City Schools (RANYCS) and conducted interviews with 25 
SSC principals and surveyed teachers in 85 (of 105) SSCs. Once analyzed, the interview data 
will provide the project with unique insight into what SSC administrators view as being 
responsible for their success. In addition, teacher responses to the SSC survey will provide the 
project with a rich dataset on SSC teachers’ school experiences that can be used to explore the 
school-level variation in SSC effects. 

Findings / Results: 

In the most recent analysis of the effects of enrolling in SSCs, Bloom and Unterman 
(2012) report that: 

• Enrolling in an SSC instead of another type of New York City public high school increased 
average four-year graduation rates by 8.6 percentage points (to 67.9 percent for target SSC 
enrollees from 59.3 percent for their control group counterparts). 5 

• Enrolling in an SSC instead of some other New York City public high school increased 
students’ college readiness in English but not math, as measured by the percentage of 
students with scores of 75 points or higher on the corresponding New York State Regents 
examinations. This threshold is used by the City University of New York to exempt students 
from taking remedial courses in these subjects. 

• The positive effect of SSC enrollment was experienced by students from all race/gender, 
prior academic achievement, free/reduced-price lunch status subgroups. 

Conclusions: 

In conclusion, the scale of New York City’s small school refonn effort and the rigor of 
the MDRC small schools study present a rare opportunity to explore the relationship between 
research and practice in New York. The proposed panelists have a variety of experiences in 
bridging these two worlds and hopefully this panel will provide them with an opportunity to 
share what they’ve learned with SREE conference attendees. 


proportion of SSC lottery winners who do not enroll in an SSC and the proportion of control group members who do 
enroll in an SSC, using a well- known statistical approach called instrumental variables analysis. 

5 The estimated average effect of SSCs on four-year graduation rates was an increase of 9.9 percentage points for the 
second study cohort as opposed to the 6.8 for the first cohort. These estimates are not statistically significantly 
different from each other and thus might differ by chance to random error. 
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